PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer

Most research on pseudo relevance feedback (PRF) has been done in vector space and probabilistic retrieval models. This paper shows that Transformer-based rerankers can also benefit from the extra context PRF provides. It presents PGT, a graph-based Transformer sparsifies attention between graph nodes to enable while avoiding high computational complexity of most architectures. Experiments show PGT improves upon non-PRF reranker, it is at least as accurate models use full attention, but with lower costs.